Papers with robustness analysis
RADDLE: An Evaluation Benchmark and Analysis Platform for Robust Task-oriented Dialog Systems (2021.acl-long)
Copied to clipboard
| Challenge: | Existing task-oriented dialog systems are less than satisfactory in robustness evaluation . existing systems are weak in robustity evaluation based on pre-training and fine-tuning . |
| Approach: | They propose to use a set of training examples to evaluate model generalization ability . they propose to include tasks with limited training data to favor models with strong generalization abilities . |
| Outcome: | The proposed model generalizes well with limited training data and is robust to user input across domains. |
Reasoning-Based Refinement of Unsupervised Text Clusters with LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Unsupervised methods are used to induce latent semantic structure from large text collections, yet their outputs often contain incoherent, redundant, or poorly grounded clusters. |
| Approach: | They propose a framework that leverages large language models as semantic judges to validate and restructure unsupervised clustering algorithms. |
| Outcome: | The proposed framework improves cluster coherence and human-aligned labeling quality over traditional models and representation-based baselines. |
Cross-Lingual Retrieval Augmented Prompt for Low-Resource Languages (2023.findings-acl)
Copied to clipboard
| Challenge: | Multilingual pretrained language models (MPLMs) perform strongly in cross-lingual transfer. |
| Approach: | They propose to augment context with similar sentences retrieved from a high-resource language (HRL) they find a significant correlation between cross-lingual transfer performance and similarity between high- and low-resourced languages . |
| Outcome: | The proposed model outperforms finetuning by 3.7% on three downstream tasks with multilingual parallel test sets across 10 LRLs covering 6 language families in unlabeled and labeled settings. |